The DNA methyltransferases (DNMTs) found in mammals include DNMT1, DNMT3A, and DNMT3B and are attractive targets in cancer chemotherapy. DNMT1 was the first among the DNMTs to be characterized, and it is responsible for maintaining DNA methylation patterns. A number of DNMT inhibitors have been reported, but most of them are nucleoside analogs that can lead to toxic side effects and lack specificity. By combining docking-based virtual screening with biochemical analyses, we identified a novel compound, DC_05. DC_05 is a non-nucleoside DNMT1 inhibitor with low micromolar IC50 values and significant selectivity toward other AdoMet-dependent protein methyltransferases. Through a process of similarity-based analog searching, compounds DC_501 and DC_517 were found to be more potent than DC_05. These three potent compounds significantly inhibited cancer cell proliferation. The structure-activity relationship (SAR) and binding modes of these inhibitors were also analyzed to assist in the future development of more potent and more specific DNMT1 inhibitors.